Repository navigation
Antalya 26.8 Backport of #124721 - Do not check Azure disks from the server config against remote_url_allow_hosts - #2905
Conversation
…url-allow-hosts Do not check Azure disks from the server config against remote_url_allow_hosts Conflicts committed as-is; the next commit resolves them: - src/Disks/DiskObjectStorage/ObjectStorages/ObjectStorageFactory.cpp (both modified) - tests/integration/test_merge_tree_azure_blob_storage/test.py (both modified)
Kept the PR's Azure host check and test, and dropped the parts that depend on code antalya-26.8 doesn't have.
|
Port-AL: cherry-pick conflicts resolved by AI Conflicted files:
Commits:
|
CI triage for
|
| Check | What failed | Cause | From this PR? |
|---|---|---|---|
| Stateless (amd_binary, cas s3 storage, parallel) Stateless (amd_asan_ubsan, cas s3 storage, parallel, 2/2) |
03770_min_columns_to_activate_adaptive_write_buffer |
Pre-existing on antalya-26.8 | No |
| Integration (amd_asan_ubsan, db disk, old analyzer, 3/8) | test_s3_cluster/test.py::test_graceful_shutdown |
Pre-existing / flaky | No |
| Stress test (amd_debug) | Bad cast from type DB::ColumnVector<_BitInt(8)> to DB::ColumnVector<unsigned short> in max() |
Real bug, but in code this PR does not touch | No (evidence below) |
| GrypeScanServer / Grype Scan | Parse step timed out | Infra | No |
| Regression release swarms | 1 module failed | Pre-existing | No |
| DCO | Commits have no Signed-off-by |
Process | No (code-wise) |
1. 03770_min_columns_to_activate_adaptive_write_buffer (both CAS S3 stateless lanes)
Code: 241. DB::Exception: Query memory limit exceeded: would use 50.12 MiB (attempt to allocate chunk of 2.00 MiB), maximum: 50.00 MiB. (MEMORY_LIMIT_EXCEEDED)
(in query: insert into metric_log_1 select * from generateRandom() limit 1 settings max_memory_usage = '50Mi' ...)
- Classification: pre-existing. The test's 50 MiB limit is too tight on the CAS S3 storage lanes.
- Evidence: the same test fails in the same lanes on other antalya-26.8 PRs. Antalya 26.8 Backport of #128217 - Evaluate the
PARTITION BYexpression of file and object storage writes once per INSERT, not once per block #2910 fails it inamd_asan_ubsan, cas s3 storage, 2/2and Antalya 26.8: Added upstream guard in IcebergFieldParseHelpers #2913 inamd_binary, cas s3 storage. This PR does not touch MergeTree write buffers. - Next step: nothing to do on this PR. It needs a separate fix, either a higher memory limit for this test or a
no-*tag for the CAS S3 lanes.
2. test_s3_cluster/test.py::test_graceful_shutdown
File: test_s3_cluster/test.py:2171 - in test_graceful_shutdown
assert errors == 0
E assert 1 == 0
- Classification: pre-existing / flaky.
- Evidence: the same failure shows up on Antalya 26.8 Backport of #128217 - Evaluate the
PARTITION BYexpression of file and object storage writes once per INSERT, not once per block #2910 and Antalya 26.8 - backport 120107 - Do not re-pull integration images the job already prefetched #2928 (same shard,db disk, old analyzer, 3/8) and on Antalya 26.8: Added upstream guard in IcebergFieldParseHelpers #2913 (amd_msan, 10/10). Those PRs don't touch Azure code. - Next step: safe to re-run.
3. Stress test (amd_debug): Logical error: Bad cast from type A to B (STID: 1499-22cb)
Logical error: 'Bad cast from type DB::ColumnVector<_BitInt(8)> to DB::ColumnVector<unsigned short>'.
src/Common/assert_cast.h:43: assert_cast<DB::ColumnVector<unsigned short> const&, DB::IColumn const&>
src/AggregateFunctions/SingleValueData.cpp:208:5: DB::SingleValueDataFixed<unsigned short>::set(...)
src/AggregateFunctions/SingleValueData.cpp:357:9: DB::SingleValueDataFixed<unsigned short>::setIfGreater(...)
src/AggregateFunctions/AggregateFunctionsMinMax.cpp:66: AggregateFunctionMinMax<SingleValueDataFixed<unsigned short>, false>::add(...)
- Classification: a real logical error, but not caused by this PR. A
max()aggregate whose state was created forUInt16received anInt8column. The planner or analyzer handed the aggregate the wrong argument type. - Evidence: the crash is entirely in aggregate-function and query-pipeline code. This PR only changes Azure disk/object-storage setup, the Azure
PocoHTTPClient/WriteBufferFromAzureDataLakeStorage, theDataLakeConfiguration.hURL check, and integration tests. None of that is on the stack or affects column types. Stress amd_debug passed on Antalya 26.8 Backport of #128217 - Evaluate thePARTITION BYexpression of file and object storage writes once per INSERT, not once per block #2910 and Antalya 26.8: Added upstream guard in IcebergFieldParseHelpers #2913, so this is a rare, random-query crash rather than a regular failure. I found no existing issue for it. - Next step: don't block this backport on it. It's worth a separate issue that includes STID
1499-22cband the server logs from this run, which are linked from the Stress test report.
4. GrypeScanServer / Grype Scan
##[error]The action 'Parse grype results' has timed out after 60 minutes.
- Classification: infra. The image scan itself found "0 high/critical vulnerabilities" (the
Grype Scan altinityinfra/clickhouse-server:2905-…status). Only the result-parsing step hung, and it was killed by the 60-minute step limit. The-alpineserver scan passed. - Next step: re-run the job.
5. Regression release swarms
- Classification: pre-existing. The swarms module also fails on Antalya 26.8 Backport of #128217 - Evaluate the
PARTITION BYexpression of file and object storage writes once per INSERT, not once per block #2910 and Antalya 26.8: Added upstream guard in IcebergFieldParseHelpers #2913, which are unrelated to this PR. - Next step: nothing to do on this PR.
6. DCO
- Classification: process only, not code. None of the 3 commits (
8af433fbf8cherry-pick,a46adaf2beconflict resolution,0c29382678cbase merge) has aSigned-off-bytrailer. Backports Antalya 26.8 Backport of #128217 - Evaluate thePARTITION BYexpression of file and object storage writes once per INSERT, not once per block #2910 and Antalya 26.8: Added upstream guard in IcebergFieldParseHelpers #2913 fail DCO the same way, so it isn't usually treated as blocking here. - Next step: if DCO is required, a maintainer can mark it passed in the DCO app. Adding sign-offs to existing commits would need a history rewrite and force-push, which I can't do.
Bottom line: CI shows no regression from this backport. The tests it adds or changes (test_allowed_url_from_config, test_merge_tree_azure_blob_storage, test_storage_iceberg_disks) do not appear among the failures. I'd re-run the Grype job and the db disk, old analyzer, 3/8 integration shard, and treat the remaining failures as known on antalya-26.8.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Fix server startup failure when an Azure Blob Storage disk from the server configuration has an endpoint that is not listed in
remote_url_allow_hosts(since 26.7). Such disks are no longer checked against this list, like S3 disks, also when a data lake table uses one viaSETTINGS disk = .... Disks created bydisk(...)in a query, URLs of table functions and table engines, and redirects are still checked. An existing table on adisk(...)disk whose host is no longer allowed loads with a warning and keeps reading and writing to that host, as for S3 disks; a newdisk(...)with that host is rejected. Closes ClickHouse#124431. (ClickHouse#124721 by @groeneai)Documentation entry for user-facing changes
...
CI/CD Options
Exclude tests:
Regression jobs to run: